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Abstract — This paper considers the problem of reconstructing 
sparse or compressible signals from one-bit quantized measure- 
ments. We study a new method that uses a log-sum penalty 
function, also referred to as the Gaussian entropy, for sparse 
signal recovery. Also, in the proposed method, sigmoid functions 
are introduced to quantify the consistency between the acquired 
one-bit quantized data and the reconstructed measurements. A 
fast iterative algorithm is developed by iteratively minimizing 
a convex surrogate function that bounds the original objective 
function, which leads to an iterative reweighted process that 
alternates between estimating the sparse signal and refining 
the weights of the surrogate function. Connections between the 
proposed algorithm and other existing methods are discussed. 
Numerical results are provided to illustrate the effectiveness of 
the proposed algorithm. 

Index Terms — Compressed sensing, one-bit quantization, 
Gaussian entropy, surrogate function. 

I. Introduction 

Conventional compressed sensing framework recovers a 
sparse signal x G W 1 from only a few linear measurements: 

y = Ax (1) 

where y E W n denotes the acquired measurements, A £ 
K mx " is the sampling matrix, and m <gi n. Such a problem 
has been extensively studied and a variety of algorithms 
that provide consistent recovery performance guarantee were 
proposed, e.g. [l]-[3]. In practice, however, measurements 
have to be quantized before being further processed. Moreover, 
in distributed systems where data acquisition is limited by 
bandwidth and energy constraints, aggressive quantization 
strategies which compress real-valued measurements into one 
or only a few bits of data are preferred. This has inspired recent 
interest in studying compressed sensing based on quantized 
measurements. Specifically, in this paper, we are interested in 
an extreme case where each measurement is quantized into 
one bit of information 

b = sign(y) = sign(Aa;) (2) 

where "sign" denotes an operator that performs the sign func- 
tion element-wise on the vector, the sign function returns 1 for 
positive numbers and —1 otherwise. Clearly, in this case, only 
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the sign of the measurement is retained while the information 
about the magnitude of the signal is lost. This makes an exact 
reconstruction of the sparse signal x impossible. Nevertheless, 
if we impose a unit-norm on the sparse signal, it has been 
shown [4], [5] that signals can be recovered with a bounded 
error from one bit quantized data. Besides, in many practical 
applications such as source localization, direction-of-arrival 
estimation, and chemical agent detection, it is the locations 
of the nonzero components of the sparse signal, other than 
the amplitudes of the signal components, that have significant 
physical meanings and are of our ultimate concern. Recent 
results [6] show that asymptotic reliable recovery of the 
support of sparse signals is possible even with only one-bit 
quantized data. 

The problem of recovering a sparse or compressible sig- 
nal from one-bit measurements was firstly introduced by 
Boufounos and Baraniuk in their work [7]. Following that, the 
reconstruction performance from one-bit measurements was 
more thoroughly studied [4]-[6], [8] and a variety of one- 
bit compressed sensing algorithms such as binary iterative 
hard thresholding (BIHT) [4], matching sign pursuit (MSP) 
[9], l\ minimization-based linear programming (LP) [5], and 
restricted-step shrinkage (RSS) [10] were proposed. Although 
achieving good reconstruction performance, these algorithms 
either require the knowledge of the sparsity level [4], [9] or 
are ^i-based methods that often yield solutions that are not 
necessarily the sparsest [5], [10]. In this paper, we study 
a new method that uses the Gaussian entropy-based penalty 
function for sparse signal recovery. The Gaussian entropy has 
the potential to be much more sparsity-encouraging than the 
li norm. By resorting to a bound optimization approach, we 
propose an iterative reweighted algorithm that successively 
minimizes a sequence of convex surrogate functions with its 
weights for the next iteration computed based on the current 
estimate. The proposed algorithm has the advantage that it 
does not need the cardinality of the support set, K, of the 
sparse signal. Moreover, our analysis and simulation results 
show that the proposed algorithm outperforms the ^i-type 
methods [5], [10] in identifying the support set of the sparse 
signal. 

II. One-Bit Compressed Sensing 

Since the only information we have about the original signal 
is the sign of the measurements, we hope that the reconstructed 
signal x yields estimated measurements that are consistent 
with our knowledge, that is 

sign(af x) = bi Vi (3) 
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or in other words 

haj x > Vi (4) 

where a L denotes the transpose of the ith row of the sampling 
matrix A, bi is the ith element of the sign vector b. This 
consistency can be enforced by hard constraints [5], [10] or 
can be quantified by a well-defined metric which is meant 
to be maximized/minimized [4], [9], [11]. In this paper, we 
introduce the sigmoid function to quantify the consistency 
between what we acquired and what we estimated. The metric 
is defined as 

m 

4>(x) ^^logMiiofa;)) (5) 

i=i 

where 

w 1 + exp(-x) 

is the sigmoid function. The sigmoid function, with an 'S' 
shape, approaches one for positive x and zero for negative 
x. Hence it is a good measure to evaluate the consistency 
between bi and afx. Also, the sigmoid function, differentiable 
and log-concave, is more amiable for algorithm development 
than the indicator function adopted in [4], [9], [11]. Note that 
the sigmoid function, also referred to as the logistic regression 
model, has been widely used in statistics and machine learning 
to represent the posterior class probability. 

Naturally our objective is to find x to maximize the 
consistency between the acquired data and the reconstructed 
measurements, i.e. 

m 

max <f>(x) = > log(a(biaf x)) (6) 
1=1 

This optimization, however, does not necessarily lead to 
a sparse solution. To obtain sparse solutions, a sparsity- 
encouraging term needs to be incorporated to encourage 
sparsity of the signal coefficients. The most commonly used 
sparsity-encouraging penalty function is l\ norm. An attractive 
property of the l\ norm is its convexity, which makes the l\- 
based minimization a well-behaved numerical problem. De- 
spite its popularity, l\ type methods suffer from the drawback 
that the global minimum does not necessarily coincide with the 
sparsest solution, particularly when only a few measurements 
are available for signal reconstruction. In this paper, we 
consider the use of an alternative sparsity-encouraging penalty 
function for sparse signal recovery. This penalty function, also 
referred to as the Gaussian entropy, is defined as 

n 

M*) = X>gM 2 ( 7 ) 

i=i 

where Xi denotes the ith component of the vector x. Such 
a log-sum penalty function was firstly introduced in [12] for 
basis selection. This penalty function has the potential to be 
much more sparsity-encouraging than the li norm. It can be 
readily observed that the log-sum (Gaussian entropy) penalty 
function, like Iq norm, has infinite slope at Xi = 0, Vi, which 
implies that a relatively large penalty is placed on small 



nonzero coefficients to drive them to zero. The reason why 
the Gaussian entropy is more sparsity-encouraging than li 
function will also be explained from an algorithmic perspective 
later in our paper. Using this penalty function, the problem of 
finding a sparse solution to maximize the consistency can be 
formulated as follows 

x = axgmin-L(x) 

m n 

= argmin — \og{a (biX T a i)) + A^ log \xi\ 2 (8) 

i=l i=l 

where A is a parameter controlling the trade-off between the 
degree of sparsity and the quality of consistency. 

A. Proposed Iterative Algorithm 

Due to the concave and unbounded nature of the Gaus- 
sian entropy, the objective function (O is non-convex and 
unbounded from below. Instead of resorting to the gradient 
descend method, we propose an iterative reweighted algorithm 
which provides fast convergence and guarantees that every 
iteration results in a decreasing objective function value. 
The algorithm is developed based on a bound optimization 
approach [14]. The idea is to construct a surrogate function 
Q{x\x { - t) ) such that 

Q{x\x {t) ) -L{x) > (9) 

and the minimum is attained when x = x^\ i.e. 
Q(x^\x^) = L(x^). Optimizing L(x) can therefore be 
replaced by minimizing the surrogate function Q(x\x^) 
iteratively. Suppose that 

x {t+1) =mmQ(x\x {t) ) 
We can ensure that 

L(x^) =L(4< t+1 >) - Q(x«+V\xW) + Q(x^\x^) 
<L(x«) - Q($W\xW) + Q(x^\x^) 
<L{x {t) ) - Q{x {t) \x [t) ) + Q{x [t) \x {t) ) 
=L(x {t) ) (10) 

where the first inequality follows from the fact that 
Q{x\x^) — L(x) attains its minimum when x = x^'; 
the second inequality comes by noting that Q(x \x {t) ) is 
minimized at x = x^ t+1 \ We see that, through minimizing 
the surrogate function iteratively, the objective function L(x) 
is guaranteed to decrease at each iteration. 

We now discuss how to find a surrogate function for the 
original problem (8). Ideally, we hope that the surrogate 
function is differentiable and convex so that the minimization 
of the surrogate function is a well-behaved numerical problem. 
Since the consistency evaluation term is convex, our objective 
is to find a convex surrogate function for the Gaussian entropy 
defined in (Q. An appropriate choice of such a surrogate 
function has a quadratic form and is given by 

™ / 2 \ 

f(x\xV)±Yl 7i+ 1 °g^ t) ) 2 - 1 
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It can be easily verified that 

f(x\x^)-h G (x)>0 (12) 

with the minimum attained when x = x^*\ Therefore the 
convex function f(x\x^) is a desired surrogate function for 
the Gaussian entropy ha(x). As a consequence, the surrogate 
function for the objective function L(x) is given by 

Q(x\x W ) 

= - £ log^^^a,)) + A £ ( -g- + log(xf) 2 - 1 

i=l i=l M^i ) ' 

m 

= - £ log((r(6 i K T a i )) + Aa? T D(* (i) )a; + constant (13) 

»=1 

where 

D^^^diagj^)- 2 ,.-.,^)- 2 } 

Optimizing L(a;) now reduces to minimizing the surrogate 
function Q(x\x <it ' > ) iterative ly. For clarity, the iterative algo- 
rithm is briefly summarized as follows. 

1) Given an initialization x^°\ 

2) At iteration t > 0, minimize Q(x\x^), which yields 
a new estimate x^ t+1 \ Based on this new estimate, 
construct a new surrogate function Q(x\x^ t+1 ^). 

3) Go to Step 2 if \\x {t+1) - x (t) \\ 2 > e, where e is a 
prescribed tolerance value; otherwise stop. 

Remark 1: The second step involves optimization of the 
surrogate function Q(x\x^). Since the surrogate function is 
differentiable and convex, minimization of Q(x\x^) is a well- 
behaved numerical problem. Any gradient-based search such 
as Newton's method which has a fast convergence rate can be 
used and is guaranteed to converge to the global maximum. 

Remark 2: The above algorithm results in a non-increasing 
objective function value of L(x). In this manner, the iterative 
algorithm eventually converges to a local minimum of L(x). 
It should be emphasized that the cost function L(x) is non- 
convex. Therefor it is important to choose a suitable starting 
point for the algorithm. Our simulations suggest that initial- 
izing with a;' ' = A T b usually delivers good reconstruction 
performance. Moreover, we found from our simulations that, 
despite starting from different (randomly generated) initial 
points, if the number of bits, m, is sufficiently large, the 
support of the reconstructed sparse solution is guaranteed to 
coincide with, or at least be a subset of, the true support of 
the sparse signal x. 

Remark 3: The proposed iterative algorithm can be consid- 
ered as composed of two alternating steps. First, we estimate x 
through minimizing the current surrogate function Q(x\x^'). 
Second, based on the estimate of x, we update the weights of 
the weighted l 2 norm penalty of the surrogate function. This 
alternating process finally results in sparse solutions. To see 
this, note that the weighted I2 norm of x has their weights 
specified as {{x\ ) -2 }- The penalty term has the tendency to 
decrease these entries in x whose corresponding weights are 
large, i.e., whose current estimates {x^ } are already small. 
This negative feedback mechanism keeps suppressing these 



entries until they reach machine precision and becomes zeros, 
while leaving only a few prominent nonzero entries survived 
to meet the consistency requirement. 

III. Related Work 

Our developed algorithm has a close connection with the 
FOcal Underdetermined System Solver (FOCUSS) algorithm 
[15] since the latter algorithm also uses an iterative reweighted 
approach to find sparse solutions to underdetermined systems 
y = Ax. Specifically, at each iteration, FOCUSS solves a 
reweighted 1% minimization with weights = l/|x^| p , 

where p£ [1 2]. It was also shown that when p = 2, FOCUSS 
is equivalent to a modified Newton's method minimizing the 
Gaussian entropy function. As compared with FOCUSS, our 
paper considers a more general framework in which the data 
model allows measurement errors/noise and is not confined 
to be linear, and a general connection between the sparsity- 
promoting penalty function and the iterative reweighted pro- 
cess is established through the use of the surrogate function. In 
[16], a similar iterative reweighted algorithm was proposed to 
enhance sparsity of recovered signals. The algorithm consists 
of solving a sequence of weighted l\ -minimization problems 
with its weights for the next iteration computed based on 
the current estimate. Such an algorithm, interestingly, is also 
shown closely related to the log-sum penalty function, i.e. the 
Gaussian entropy function. 

We now explore the relationship between our proposed 
algorithm and the l\ -minimization type methods. Following 
dHJ, we can formulate a l\ norm-based sparsity -promoting 
optimization 

x = argminL(a;) 

m n 

= argmin-£log(a(&ia7 T a i )) + A^ \xi\ (14) 

z=l i—l 

Similarly, a surrogate function can be constructed to bound the 
above optimization, from which an iterative algorithm can be 
developed to solve ( fT4l . Note that the l\ function \x\ can also 
be upper bounded by a quadratic function. It can be readily 
verified that the surrogate function for l\ norm is given by 

/>l^)^£(^ + l#l) d5) 

z i=i \R I / 

Therefore the solution to ( fl4l ) can be found by iteratively 
minimizing the following surrogate function 

Q(x\x {t) ) 

m . n / 2 \ 

= -E 1 ^(^)) + 5E(^k + i^ ) i) 

i=l i=l \ \ x i ' 

m , 

= — £ \og(a(biX T ai)) + -x T D(x^)x + constant (16) 
where 

6,£ "^ d,a 4^-'sk} 
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The surrogate function Q(x\x^) has a similar form as (fT~3T >, 
except that the surrogate function for the Gaussian entropy 
updates its weights using (x^) -2 , while the surrogate func- 
tion for the li norm updates its weights with (|a^* This 
seemingly slight difference, however, results in very different 
convergence behavior. The latter update equation guarantees 
converging to a unique global minimum. On the other hand, 
the update method used in our algorithm renders a more 
intense effect in de-emphasizing the entries in x. Therefore 
the proposed algorithm is more sparsity-encouraging than the 
li type methods, and generally yields a more sparse solution. 

IV. Numerical Results 

We now carry out experiments to illustrate the performance 
of our proposed one-bit compressed sensing algorithm. In 
our simulations, the Jf-sparse signal is randomly generated 
with the support set of the sparse signal randomly chosen 
according to a uniform distribution. The signals on the support 
set are independent and identically distributed (i.i.d.) Gaussian 
random variables with zero mean and unit variance. The 
measurement matrix A £ W nxn is randomly generated with 
each entry independently drawn from Gaussian distribution 
with zero mean and unit variance, and then each column of A 
is normalized to unity for algorithm stability. We compare our 
proposed algorithm with the other two algorithms, namely, the 
li minimization-based linear programming (LP) algorithm [5] 
(referred to as "one-bit LP"), and the iterative algorithm pro- 
posed in Section Hn] to solve the optimization (fl4l > (referred to 
as "L 1 -optimization (TBlT). Both the latter two algorithms are 
li type methods, with the consistency evaluated by different 
criteria. 

We investigate the support recovery performance of respec- 
tive algorithms. Support recovery accuracy are measured by 
the false alarm (misidentified) rate and the miss rate. A false 
alarm event represents the case where coefficients that are 
zero in the original signal are misidentified as nonzero after 
reconstruction, while a miss event stands for the case where 
the nonzero coefficients are missed and determined to be zero. 
Fig. Q] depicts the false alarm and miss rates of respective 
algorithms as a function of the sparsity level K, where we 
set m = 100, n = 50, and A = 1/2 in our simulations. 
Results are averaged over 10 4 independent runs. We see that 
the proposed algorithm provides more accurate identification 
of the true support set: it has a similar (or slightly higher) miss 
rate as (than) that of the Zi-based methods, while meanwhile 
achieves a considerably lower false alarm rate than both l± type 
methods. Our result also indicates that the proposed algorithm 
yields solutions more sparse than l\ type methods, which 
corroborates our theoretical analysis. 

V. Conclusions 

We proposed an iterative reweighted algorithm for sparse 
signal reconstruction from one-bit quantized measurements. 
The proposed algorithm consists of solving a sequence of 
minimization problems whose weights are updated based on 
the current estimate. Analyses and simulation results show 
that the proposed algorithm that uses the log-sum penalty 
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Fig. 1. (a). False alarm rates of respective algorithms; (b). Miss rates of 
respective algorithms. 

function is more sparsity-encouraging than Zi-based methods, 
and outperforms l\ type methods in recovering the true support 
of the sparse signal. 
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